Deploy to Azure Container Instance - “Bring your own container”

Following this tutorial

Steps:

  1. Setup API and containerise
  2. Deploy model
  3. Test model
  4. Delete endpoint

Model to be deployed:

model_path = "../../models/model.pkl"
resource_group = "azure-cognitive-services-accelerator"

1. Setup

First, we create an image for a Docker container that wraps our model. Then, we create an Azure Container Registry (ACR) resource and push our Docker image to it.

1.1 Create inference API

This can be infinitely configured. Here, inference is performed on the scikit-learn model using a simple FastAPI REST API which exposes the endpoint on /. For example, you could add load balancing by passing traffic through nginx.

%%writefile requirements.txt
numpy
scikit-learn==1.2.0
joblib
fastapi
uvicorn
pydantic
%%writefile app.py
from fastapi import FastAPI
import joblib, json
import numpy as np
import sklearn
from pydantic import BaseModel

class InputData(BaseModel):
    data: list

app = FastAPI()

@app.post("/")
async def root(inputData: InputData):
    print(inputData)
    pred = joblib.load('model.pkl').predict(np.array(inputData.dict()['data']))
    return {"prediction": pred.tolist()}

1.2 Containerise application

The Dockerfile gives Docker instructions on how to setup your Docker container’s image and how it should behave at runtime. We use uvicorn to expose the API.

%%writefile Dockerfile
FROM python:3.10
WORKDIR /code
COPY . /code
RUN pip install -r /code/requirements.txt --trusted-host pypi.org --trusted-host files.pythonhosted.org
CMD ["uvicorn","app:app","--host", "0.0.0.0", "--port", "80"]

Start Docker locally and run the following command to build your image in the current folder. Docker can be downloaded from here.

!docker build -t sample_project:0.1 .

1.3 Create ACR resource

!az acr create --resource-group $resource_group --name sampleprojectacr --sku Basic
!az acr login --name sampleprojectacr

Get full name of registry login server

!az acr show --name sampleprojectacr --query loginServer --output table

1.4 Push Docker image

Tag Docker image with details of registry server

!docker tag sample_project:0.1 sampleprojectacr.azurecr.io/sample_project:0.1

Push Docker image to the ACR

!docker push sampleprojectacr.azurecr.io/sample_project:0.1

Enable admin access to ACR as per here

!az acr update -n sampleprojectacr --admin-enabled true

2. Deploy image from ACR to a container instance on ACI

You could do this programmatically with az container create as per the Microsoft tutorials, but this requires configuring credentials of the registry login account. For the demo, it’s much easier to do from the Azure Portal browser interface under Container Instances. Click Create, and set:

  • Resource group: same as above
  • Container name: new unique name
  • Region: uksouth
  • Image source: Azure Container Registry
  • Registry: sampleprojectacr
  • Image + Image tag: sampleprojectacr.azurecr.io/sample_project:v1
  • Optional: set DNS name label under “Networking” for custom URL
container_name = "sampleprojectaci"

3. Test endpoint

Find container endpoint IP from Azure portal, or access using the command line:

endpoint_url = !az container show --name $container_name --resource-group $resource_group --query ipAddress.ip
endpoint_url = "http://" + endpoint_url[0][1:-1]
import requests
import json
import numpy as np

input_payload = json.dumps({
    'data': np.load("../data/diabetes.npz")["X_test"][:2].tolist(),
})

requests.post(endpoint_url, input_payload, headers={'Content-Type':'application/json'}).json()

4. Delete container and registry

!az container delete --name $container_name --resource-group $resource_group --yes
!az acr delete --name sampleprojectacr --resource-group $resource_group --yes
!rm app.py
!rm Dockerfile
!rm requirements.txt